IN THE CLAIMS: 



1-6. (cancelled) 

7. (currently amended) Th e proc e ss of Claim 6, wh e r ei n th e fus i ng 
n e ura l n e twork A computer-implemented face recognition process for identifying 
a person depicted in an input image, comprising using a computer to perform the 
following process actions: 

creating a database of a plurality of model image characterizations, 
each of which represents the face of a known person that it is desired to identify 
in the input image as well as the person's face pose; 

training a neural network ensemble to identify a person and their 
face pose from a region which has been extracted from said input image and 
characterized in a manner similar to the plurality of model images, wherein the 
network ensemble comprises, a first stage having a plurality of classifiers each of 
which has input and output units and is dedicated to a particular pose range and 
outputs a measure of the similarity indicative of the similarity between said 
characterized input image region and each of said model image characterizations 
associated with the particular pose range of the classifier and a fusing neural 
network as its second stage which combines the outputs of the classifiers to 
generate an output indicative of the person associated with the characterized 
input image region and the face pose of that person and which has at least 
enough output units to allow a different output to represent each person it is 
desired to identify at each of the pose ranges, and wherein th e proc e ss act i on of 
tra i n i ng th e n e ural n e twork e ns e mbl e furth e r compris e s th e act i ons of: training 
the neural network ensemble comprises. 

preparing each model image characterization from a model 
image depicting the face of a known person that it is desired to identify in the 
input image by, 

extracting the portion of the model image depicting 

said face. 
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normalizing the extracted portion of the model image 
by resizing it to a prescribed scale if not already at the prescribed scale and 
adiusting the region so that the eye locations of the depicted subject fall within a 
prescribed area, and 

cropping the extracted portion of the model image by 
eliminating unneeded portions of the image not specifically depicting part of the 
face of the subiect to create a model face image, 

categorizing the model face images by assigning each to 
one of a set of pose ranges into which its associated face pose falls, and 
for each pose range, 

choosing a prescribed number of the model face 
images of each person being modeled which have been assigned to the selected 
pose range, 

concatenating each of the chosen model face images 
to create a respective dimensional column vector (DCV) for each, 

computing a covariance matrix from the DCVs, 
calculating eigenvectors and corresponding 
eigenvalues from the covariance matrix, 

ranking the eigenvalues in descending order, 
identifying a prescribed number of the top 

eigenvalues, 

using the eigenvectors corresponding to the identified 
eigenvalues to form the rows of a basis vector matrix (BVM) for the pose range, 
and 

multiplying each DCV by each BVM to produce a set 
of principal components analysis (PCA) coefficient vectors for each model face 
image, and 

for each face recognition neural network, inputting, one at a 
time, each of the PCA coefficient vectors associated with the pose range of the 
face recognition neural network into the inputs of the network until the outputs of 
the network stabilize^i 
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initializing the fusing neural network for training^f 
for each DCV, simultaneously inputting the PCA coefficient 
vectors generated from the DCV into the respective face recognition neural 
network associated the vector's particular pose range group until all the PCA 
coefficient vectors of every DCV have been input, and repeating until the outputs 
of the fusing neural network stabilize^f and 

for each DCV, simultaneously inputting the PCA coefficient 
vectors generated from the DCV into the respective face recognition neural 
network associated the vector's particular pose range group and assigning the 
active output of the fusing neural network as corresponding to the particular 
person and pose associated with the model image used to create the set of PCA 
coefficient vectors : and 

employing the network ensemble to identify the person associated 
with the characterized input image region and the face pose of that person . 

8. (previously presented) The process of Claim 7, wherein the process 
action of employing the neural network ensemble to identify the person depicted 
in the input image face region, comprises the actions of: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficient vector in the set of PCA coefficient 
vectors into the respective face recognition neural network associated that 
vector's particular pose range group; and 
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identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with the extracted face region. 

9-16. (cancelled) 

17. (currently amended) Th e systom of Claim 16, whoroin th o fus i ng 
n e ura l network A face recognition svstem for identifying a person depicted in an 
input image, comprising: 

a general purpose computing device: and 
a computer program comprising program modules executable bv 
the computing device, wherein the computing device is directed by the program 
modules of the computer program to. 

capture model images, each of which depicts at least one 
person of known identity. 

locate and extract regions within the model images, each of 
which depicts the face of a known person that it is desired to identify in the input 
image. 

determine the face pose for each of the face regions 
extracted from the model images. 

categorize each face region by assigning each to one of a 
set of pose ranges into which its associated face pose falls. 

train a neural network ensemble to identify a person and 
their face pose from a region that depicts the face of a person which has been 
extracted from said input image, wherein the network ensemble comprises, a first 
stage having a plurality of classifiers each of which has input and output units 
and is dedicated to a particular pose range and outputs a measure of the 
similarity indicative of the similarity between said input image region and each of 
said model image regions associated with the particular pose range of the 
classifier, and a fusing neural network as its second stage which combines the 
outputs of the classifiers to generate an output indicative of the person 
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associated with the characterized input image region and the face pose of that 
person and which has at least enough output units to allow a different output to 
represent each person it is desired to identify at each of the pose ranges, and 
wherein th e sub modul e for train i ng th o noura l network e nsembl e further 
compr i s o s sub modul e s for: training the neural network ensemble comprises. 

(a) preparing each face region extracted from said 
model images bv normalizing and cropping the extracted regions, wherein said 
normalizing comprises resizing each extracted face region to the same 
prescribed scale if not already at the prescribed scale and adjusting each region 
so that the eve locations of the depicted subject fall within the same prescribed 
area, and wherein said cropping comprises eliminating unneeded portions of the 
image not specifically depicting part of the face of the subject. 

(b) selecting a previously unselected one of the 

set of pose ranges. 

(c) choosing a prescribed number of the prepared 
face images of each person being modeled which have been assigned to the 
selected pose range. 

(d) concatenating each of the chosen prepared 
face images to create a respective dimensional column vector (DCV) for each. 

(e) computing a covariance matrix from the DCVs. 

(f) calculating eigenvectors and corresponding 
eigenvalues from the covariance matrix. 

(g) ranking the eigenvalues in descending order. 

(h) identifying a prescribed number of the top 

eigenvalues. 

(i) using the eigenvectors corresponding to the 
identified eigenvalues to form the rows of a basis vector matrix f BVM) for the 
selected pose range. 

(j) repeating actions (b) through (i) for each 

remaining pose range, 

(k) multiplying each DCV by each BVM to produce 
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a set of principal components analysis (PCA) coefficient vectors for each face 
image 

(!) for each face recognition neural network, 
inputting, one at a time, each of the PCA coefficient vectors associated with the 
pose range of the face recognition neural network Into the inputs of the network 
until the outputs of the network stabilize^y 

(nil Initializing the fusing neural network for 

training^i 

M for each DCV, simultaneously inputting the 
PCA coefficient vectors generated from the DCV into the respective face 
recognition neural network associated the vector's particular pose range group 
until all the PCA coefficient vectors of every DCV have been Input, and repeating 
until the outputs of the fusing neural network stabilize^j and 

i&i for each DCV, simultaneously Inputting the 
PCA coefficient vectors generated from the DCV into the respective face 
recognition neural network associated the vector's particular pose range group 
and assigning the active output of the fusing neural network as corresponding to 
the particular person and pose associated with the model Image used to create 
the set of PCA coefficient vectors , and 

emplov the network ensemble to identify the person 
associated with the characterized Input image region and their face pose . 

18. (original) The system of Claim 17, wherein the sub-module for 
employing the neural network ensemble to identify the person depicted In the 
Input Image face region and the pose associated with the face of the identified 
person, comprises sub-modules for: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale If not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
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comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficient vector in the set of PCA coefficient 
vectors into the respective face recognition neural network associated that 
vector's particular pose range group; and 

identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with the extracted face region. 

19-26 (cancelled). 

27. (currently amended) Th e comput e r r e adabl e m e mory of Claim 26, 
wh e r e in th e fusing n e ural n e twork A computer-readable memory for use in 
identifying a person depicted in an input image, comprising: 
a computer-readable storage medium: and 
a computer program comprising program modules stored in the 
storage medium, wherein the storage medium is so configured by the computer 
program that it causes a computer to. 

input model images, each of which depicts at least one 
person of known identity, 

locate and extract regions within the model images, each of 
which depicts the face of a known person that it is desired to identify in the input 
image, 

determine the face pose for each of the face regions 
extracted from the model images. 

categorize each face region by assigning each to one of a 
set of pose ranges into which its associated face pose falls. 

train a neural network ensemble to identify a person and 



8 



their face pose from a region that depicts the face of a person which has been 
extracted from said input image, wherein the network ensemble comprises, a first 
stage having a plurality of classifiers each of which has input and output units 
and is dedicated to a particular pose range and outputs a measure of the 
similarity indicative of the similarity between said input image region and each of 
said model image regions associated with the particular pose range of the 
classifier and a fusing neural network as its second stage which combines the 
outputs of the classifiers to generate an output indicative of the person 
associated with the characterized input image region and the face pose of that 
person and which has at least enough output units to allow a different output to 
represent each person it is desired to identify at each of the pose ranges, and 
wherein th e sub modul e for training tho noural network ons o mb i o further 
compris e s sub modu le s for: training the neural network ensemble comprises. 

(a) preparing each face region extracted from said 
model images by normalizing and cropping the extracted regions, wherein said 
normalizing comprises resizing each extracted face region to the same 
prescribed scale if not already at the prescribed scale and adiusting each region 
so that the eye locations of the depicted subject fall within the same prescribed 
area, and wherein said cropping comprises eliminating unneeded portions of the 
image not specifically depicting part of the face of the subject. 

(b) selecting a previously unselected one of the 

set of pose ranges, 

(c) choosing a prescribed number of the prepared 
face images of each person being modeled which have been assigned to the 
selected pose range. 

(d) concatenating each of the chosen prepared 
face images to create a respective dimensional column vector (DCV) for each, 

(e) computing a covariance matrix from the DCVs. 

(f) calculating eigenvectors and corresponding 
eigenvalues from the covariance matrix, 

(g) ranking the eigenvalues in descending order. 
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(h) identifying a prescribed number of the top 

eigenvalues. 

(1) using the eigenvectors corresponding to the 
identified eigenvalues to form the rows of a basis vector matrix (BVM) for the 
selected pose range. 

(i) repeating actions (b) through (i) for each 

remaining pose range. 

(k) multiplying each DCV bv each BVM to produce 
a set of principal components analysis fPCA) coefficient vectors for each face 
image 

(Jl for each face recognition neural network, 
inputting, one at a time, each of the PCA coefficient vectors associated with the 
pose range of the face recognition neural network into the inputs of the network 
until the outputs of the network stabilize^i 

(m) initializing the fusing neural network for 

training^i 

M for each DCV, simultaneously inputting the 
PCA coefficient vectors generated from the DCV into the respective face 
recognition neural network associated the vector's particular pose range group 
until all the PCA coefficient vectors of every DCV have been input, and repeating 
until the outputs of the fusing neural network stabilize^r and 

(oi for each DCV, simultaneously inputting the 
PCA coefficient vectors generated from the DCV into the respective face 
recognition neural network associated the vector's particular pose range group 
and assigning the active output of the fusing neural network as corresponding to 
the particular person and pose associated with the model image used to create 
the set of PCA coefficient vectors , and 

employ the network ensemble to identify the person 
associated with the characterized input image region and their face pose . 

28. (original) The computer-readable memory of Claim 27, wherein the 
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sub-module for employing the neural network ensemble to identify the person 
depicted in the input image face region and the pose associated with the face of 
the identified person, comprises sub-modules for: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficient vector in the set of PCA coefficient 
vectors into the respective face recognition neural network associated that 
vector's particular pose range group; and 

identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with the extracted face region. 

29-31. (cancelled) 

32. (currently amended) Th e procooc of C l aim 31, whoro i n tho procooG 
action of train i ng the n e twork c nsomb l o further compr i s o o th o actions of:_ A 
computer-implemented face recognition process for identifying a person depicted 
in an input image, comprising using a computer to perform the following process 
actions: 

creating a database of a plurality of model image characterizations, 
each of which represents the face of a known person that it is desired to identify 
in the input image as well as the person's face pose: 

training a neural network ensemble to identify a person and their 
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face pose from a region which has been extracted from said input image and 
characterized in a manner similar to the plurality of model images, wherein the 
network ensemble comprises, a first stage having a plurality of classifiers each of 
which has input and output units and is dedicated to a particular pose range and 
outputs a measure of the similarity indicative of the similarity between said 
characterized input image region and each of said model image characterizations 
associated with the particular pose range of the classifier, and a fusing neural 
network as its second stage which combines the outputs of the classifiers to 
generate an output indicative of the person associated with the characterized 
input image region and the face pose of that person, and wherein training the 
network ensemble comprises. 

deriving each model image characterization from a set of 
model images of people where each model image of the same person shows that 
person at a different face pose, said deriving comprising . 

extracting the portion of each model image depicting 

a face. 

normalizing the extracted portion of each model 
image bv resizing it to a prescribed scale if not already at the prescribed scale 
and adjusting the region so that the eve locations of the depicted subiect fall 
within a prescribed area. 

cropping the extracted portion of each model image 
by eliminating unneeded portions of the image not specifically depicting part of 
the face of the subiect to create a model face image. 

concatenating each of the model face images to 
create a respective model dimensional column vector (DCS/) for each. 

categorizing the model DCVs bv assigning each to 
one of a set of pose ranges into which its associated face pose falls, and 

inputting the model DCV of the each model face 
image falling in a particular pose range, one at a time, to a pre-selected classifier 
dedicated to the particular pose range. 

initializing the fusing neural network for training^f 



12 



simultaneously inputting the respective DCV of each model 
face image into all classifiers, until the DCV of every model image has been 
input, and repeating until the outputs of the neural network stabilizej.i and 

simultaneously inputting the respective DCV of each model 
face image into all classifiers, and assigning the active output the neural network 
as corresponding to the particular person and pose associated with the model 
image used to create the DC V: and 

employing the network ensemble to identify the person associated 
with the characterized input image region and the face pose of that person. 

33. (previously presented) The process of Claim 32, wherein the 
process action of employing the network ensemble to identify the person 
depicted in the input image face region, comprises the actions of: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

inputting the DCV of the face region into all classifiers; and 

identifying the active output of the neural network and designating 
the person previously assigned to that unit as the person associated with the 
extracted face region. 

34. (original) The process of Claim 33, further comprising a process 
action of specifying that the person designated as associated with the extracted 
face region has the face pose previously assigned to the identified active output. 

35-42. (cancelled) 
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